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(54) INPUT DEVICE AND METHOD AND STORAGE MEDIUM 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide input device and 
method and a storage medium capable of performing an 
operation with one hand without the need of selecting 
the plural pieces of icons one by one in an order while 
pressing a shift key on a keyboard in the case of 
selecting them. 

SOLUTION: By simultaneously bringing plural fingers into 
contact with a touch panel screen and utilizing the track 
patterns as gestures, the objects such as the plural 
pieces of the icons are simultaneously selected and 
operated with one hand, the plural pieces of the edges of 
a window are selected and size is changed 
simultaneously in plural directions. Also, the contacting 
orders of the respective fingers obtained at the time of 
bringing the plural fingers into contact with a display 
screen are detected and the kind of an argument is 
detected by the difference of the contacting order. By 
dragging and dropping the finger specifying the argument 
by the contacting order to the icon of a program 
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requiring the plural arguments, the program specifying the argument is executed with one 



* NOTICES * 



JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]A detection means to be an input device which carries out direct contact of a tablet or 
the touch-panel screen, and directs them, and to detect two or more points of contact A turn 
detection means to detect turn of contact of two or more points of contact detected by this 
detection means, An input device having a creating means which generates an output instruction 
for directing execution of a program judged by program judging means which Judges a program 
compatible with turn of contact detected by this turn detection means, and this program judging 
means. 

[Claim 2]The input device according to claim 1 judging a program which said program judging 
nneans has a means to distinguish a kind of argument by difference in contacting orders, is 
distinguished by this means, and needs two or more arguments based on an argument. 
[Claim 3]The input device according to claim 1 said program judging means's assigning a different 
input mode according to the number of contact, and changing to a corresponding input mode. 
[Claim 4]It is an input method which carries out direct contact of a tablet or the touch-panel 
screen, and directs a position, An input method generating an output instruction for detecting 
two or more points of contact, detecting turn of contact of two or more of said detected points 
of contact, judging a program compatible with turn of said detected contact, and directing 
execution of a judged program. 

[Claim 5]It is the storage which memorized a program which recognizes as input information 
directed by carrying out direct contact of a tablet or the touch-panel screen, and in which 
computer reading is possible, A storage which memorized a program which makes an output 
instruction for directing execution of a program which made judge a program compatible with 
turn of said contact which made detect turn of contact of two or more of said points of contact 
which made detect two or more points of contact, and were detected, and was detected, and 
was judged generate. 



[Translation done.] 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to the input device, method, and storage using two 
or more points of contact which operate objects, such as written data and an icon, by contacting 
a display screen with a finger. 
[0002] 

[Description of the Prior Art]The touch panel is provided with the inputting function which can 
choose objects displayed on the screen, such as an icon and a window, or can draw a 
handwritten character, an illustration, etc. by touching a screen directly with a finger or a pen 
with the display function as a computer display. A finger and a pen can substitute the role of the 
mouse in the conventional desktop computer for a touch panel. 

[0003]It roughly divided into the conventional touch panel, and there were the following two kinds 
of input modes in it. 

[0004]The first input mode is writing mode which writes a character with a finger or draws an 
illustration. The second input mode is the gesture mode in which select an icon with a finger, 
draw the easy form with a finger in carrying out a click and drag and drop, and operation 
corresponding to the motion is performed. There was a thing that a character string will be 
deleted as an example of gesture if a RE seal is written from on a character string by editors, 
such as a memo pad, or the object which is contained in the enclosure if an icon and a character 
string are surrounded by O was chosen etc. 

[0005] . , ^ r 

[Problem to be solved by the invention]It assumed that these conventional gesture drew a line 
with one pen or finger. Therefore, when two or more icons were selected, for example, it had to 
presuppose that it chooses one at a time in order etc., pressing the Shift key by a keyboard, and 
was not able to be operated single hand. Since only one chose the end of the window when the 
size of a window was changed, it has changed in size only into one way at once. 
[0006] 

[Means for solving problem]In order to solve such a problem, as for this invention, the invention 
in this application is characterized by that the input device which carries out direct contact of a 
tablet or the touch-panel screen, and directs them comprises the following. 
A detection means to detect two or more points of contact. 

A turn detection means to detect the turn of contact of two or more points of contact detected 
by this detection means. 

The program judging means which judges a program compatible with the turn of the contact 
detected by this turn detection means. 

The creating means which generates the output instruction for directing execution of the 
program judged by this program judging means. 

[0007]The program which said program judging means has a means to distinguish the kind of 
argument by the difference in contacting orders, is distinguished by this means, and needs two 
or more arguments based on an argument is judged. 



[0008]Said program judging means assigns a different input mode according to the number of 
contact, and it changes to a corresponding input mode. 

[0009]It is an input method which carries out direct contact of a tablet or the touch-panel 
screen, and directs a position, The output instruction for detecting two or more points of 
contact, detecting the turn of contact of two or more of said detected points of contact, judging 
a program compatible with the turn of said detected contact, and directing execution of the 
judged program is generated. 

[0010]It is the storage which memorized the program which recognizes as input the information 
directed by carrying out direct contact of a tablet or the touch-panel screen, and in which 
computer reading is possible, It is the storage which memorized the program which makes the 
output instruction for directing execution of the program which made judge a program compatible 
with the turn of said contact which made detect the turn of contact of two or more of said 
points of contact which made detect two or more points of contact, and were detected, and was 
detected, and was judged generate. 

[0011]That is, by contacting two or more fingers on a touch-panel screen simultaneously, and 
using those locus patterns as gesture in this invention. It becomes possible to choose objects, 
such as two or more icons, simultaneously single hand, to operate them, or to choose two or 
more ends of a window and to change in size in two or more directions at once. 
[0012]The contacting orders of each finger obtained when contacting two or more fingers to a 
display screen are detected. The program which specified the argument can be executed single 
hand by drags and drops the finger which could distinguish the kind of argument by the 
difference in contacting orders, and specified the argument by contacting orders to the icon of 
the program which needs two or more arguments. 

[0013]An input mode can be changed single hand by distinguishing an input mode (writing mode, 
gesture mode) by the difference in the number of a finger in contact with a screen, without using 
a menu, a button, etc. 
[0014] 

[Mode for carrying out the invention]Hereafter. an embodiment of the invention is described, 
referring to Drawings (the first working example: example of the gesture using two or more points 
of contact.). Argument specification is carried out with two or more fingers, and it dragging and 
dropping to a program icon. 

[0015]In the first working example, by assigning the argument from which a meaning differs to 
each finger according to the turn in contact with a touch panel, and drags and drops those 
fingers to the program (icon) which needs two or more arguments describes the example which 
executes the program which specified the argument. For example, when there is a program which 
enciphers the file A by the key written to the file B. (1) Two arguments which specified two 
fingers which specified the specification (2) file B with the finger of one flat knot, and specified 
the specification (3) file A and the file B for the file A with the finger of two flat knots as the 
enciphered program when dragged and dropped to the icon of the enciphered program are 
passed, and encryption is performed. 

[0016]Used an input method of this invention, for example, drawing 1 is a functional block 
diagram of a personal computer, a notebook sized personal computer, and other information 
processing equipments. The coordinate input parts 1 are input devices, such as a touch panel 
device, and detect heat, a pressure, etc. of a hand of a user who contacted. 
[0017]The coordinate information sensing device part 2 detects the number and a locus of a 
point of contact from coordinate data obtained from the device-for-inputting-coordinates part 1. 
and detects further coordinate information, such as contacting orders showing whether it is that 
by which each point of contact was contacted by what position. It is judged whether inputted 
coordinates are the things showing a coordinate information setting-out start. 
[0018]The coordinate information treating part 3 processes distinguishing an argument and a 
program which are inputted according to the number and contacting orders of a point of contact, 
or track data etc. The coordinate information presentation part 4 performs processing which 
shows a user information, including a coordinate value which a user has pointed out. a parameter 
identifier and an argument value which are obtained from the coordinate information primary 



detecting element 2 or the coordinate information treating part 3, a program name which it is 
going to perform, etc. Presentation of coordinate information is performed when judging a 
program executed in the coordinate information treating part 3. 

[0019]The coordinate information set part 5 performs processing which sets up information 
required for processing by the coordinate information treating part 3. 

[0020]It is the operating system having contained GUI, and 0S6 answers the Data Processing 
Division made by the coordinate information treating part 3, and a screen display is changed or it 
takes information from a screen. There are MS-Windows of Microsoft Corp., etc. as an example 
of the operating system containing GUI. This is used when performing processing performed by 
the external storage, the memory, CPU, and the coordinate information treating part 3. An input 
device points out input devices other than a touch panel device, and contains a keyboard, a 
mouse, etc. A display device contains a touch panel device, a computer display or a speaker of 
an input-and-output integral type, etc. 

[0021] Drawing 2 shows an example of the coordinate information primary detecting element 2 in 
the first working example. The hand primary detecting element 1 1 detects heat, a pressure, etc. 
of a hand in contact with the coordinate input part 1 , and stores in the hand pattern storage 1 2 
the position coordinate to which the user's hand contacted every fixed time deltat. It is judged 
whether the inputted coordinates are the things showing a coordinate information setting-out 
start. 

[0022]The coordinates showing the position of the hand of the user in contact with the touch 
panel obtained in the hand primary detecting element 1 1 are stored in the hand pattern storage 
12. The representation coordinates primary detecting element 13 detects the coordinates 
representing the number of a finger in contact with a touch panel, and the point of contact of 
each finger from the pattern stored in the hand pattern storage 12, and stores in the 
representation coordinates storage 14. 

[0023]The coordinates showing the point of contact of each finger obtained in the representation 
coordinates primary detecting element 13 and a touch panel are stored in the representation 
coordinates storage 14. 

[0024]In the contacting-orders judgment part 15, the coordinates of the representation 
coordinates storage 14 are rearranged according to the turn of having been contacted, and it 
stores in the contacting-orders storage 16. The result obtained by the contacting-orders 
judgment part 15 is stored in the contacting-orders storage 16. 

[0025]An example of the hand pattern storage 12 is shown in drawing 3 . It is considered as a 
two-dimensional procession with the number of elements doubled with the resolution of the 
touch panel, if in contact with the dot corresponding to each element of a procession — the 
contact flag 1 — if not in contact, the non-contact flag 0 is stored. For example, in the touch 
panel whose resolution is a horizontal Wx length H dot, dra wing 3 means that it is in contact with 
coordinates (3, 1) (3, 2) (2. 2) (4, 2) (5, 2) (2, 3) (3, 3) (4, 3). 

[0026]An example of the detecting method of the representation coordinates in the 
representation coordinates primary detecting element 1-3 is explained using drawing 4 . Drawing 
A_ia) is one of the examples of composition of the representation coordinates primary detecting 
element 13. The group division of the group dividing part 31 is carried out so that adjoining 
coordinates may become the same group about all the coordinates in which the contact flag 1 is 
stored by the hand pattern storage 12. As an example of a group division, it is considered by 
drawing 4 (b) that those by which the contact flag 1 is contained in either of the eight directions 
of adjoining four directions, the diagonal right, the diagonal left, the diagonal below, and the 
diagonal below are the same groups. 

[0027]The group division result calculated by the group dividing part 31 is stored in the group 
storage 32. How to store group names (1, 2, etc.) in the procession value corresponding to each 
coordinates like drawing 4 (c) as an example of the group storage 32 can be considered. 
[0028]The representation coordinates calculation part 33 searches for each group's 
representation coordinates. It is good also considering the centroid value of all the coordinates 
belonging to a group as representation coordinates, and good also considering the center value 
of a lengthwise direction and a transverse direction as representation coordinates. The 



representation coordinates searched for in this way are stored in the representation coordinates 
storage 14. 

[0029]An example of the representation coordinates storage 14 is explained using drawing 5 . A 
coordinate value is stored in the representation coordinates storage 14 at the order for which it 
asked in the representation coordinates primary detecting element 13. For example, it means 
that (3, 2) are stored as the group s 1 representation coordinates, and (638, 2) are stored as the 
group's 2 representation coordinates in drawing 5 . 

[0030] Hereafter, although m coordinates (Pi.x, Pi.y) (1 <=i<=m) shall be stored in the 
representation coordinates storage 14 for explanation, an explanation top (Pi.x, Pi.y) may be 
abbreviated to Pi, and may be described. 

[0031]The contacting-orders storage 16 is explained using drawing 6 . The result of having 
rearranged the coordinate value of the representation coordinates storage 14 into the turn which 
contacted is stored in the contacting-orders storage 16. The result of having judged whether it 
being released by it being in contact from each coordinates being contacted for the first time at 
present or before deltat or this time is stored as a flag. 

[0032]for example, — if are contacted for the first time and it is in contact from before 1 and 
deltat — 2 -1 is stored if released. Furthermore, the information on coordinates that it was 
contacted in front of 1 time that is, from this time, and was in contact in front of deltat from this 
time is also stored in the contacting-orders storage 16. 

[0033]The example of drawing 6 means that each coordinates were contacted as new 
coordinates in the turn of the 1st (80,100), the 2nd (480,200), and the 3rd (140,300) at the time 
before deltat. further — this time ~ the finger of (80,100) ~ a few ~ shifting (90, 90) — 
becoming — the finger of (140,300) — a few — shifting (150.330) — become and contact of the 
finger of (480,200) is released — being new (600,360) — it means having been contacted. 
[0034]Hereafter, although n coordinates (Qj.x, Qj.y) and flag Qj.flg(s) (1 <=j<=n) shall be stored in 
the contacting-orders storage 1 6 for explanation, the thing on explanation (Glj.x, G^.y) is 
abbreviated to Qj, and the thing of Qj.flg may be abbreviated to flgOj and may be described. It is 
assumed that Qlj and fIgQIj express information at present, and QOj and fIgQOj express a 1-time 
front that is, from this time, and they express the information in front of deltat from this time. 
[0035]Next, an example of the judgment method of the contacting orders in the contacting- 
orders judgment part 15 is explained. In the contacting-orders judgment part 15, the coordinate 
value of the representation coordinates storage 14 is rearranged into the turn which contacted. 
Rearrangement according to contacting orders is performed by comparing the coordinates Pi of 
the representation coordinates storage 1-4 showing the point of contact in this time with the 
coordinates QOj of the contacting-orders storage 16 showing the point of contact in front of 
deltat. It judges whether it is specifically in contact from that the coordinates of the 
representation coordinates storage 14 are contacted for the first time at present, or before 
deltat, and it is performed. 

[0036] Hereafter, the flow of processing is explained using drawing 7 . First, about each 
coordinates Pi of the representation coordinates storage 14, out of the coordinates QOj stored in 
the contacting-orders storage 16, the coordinates in the nearest distance are chosen from Pi, 
and distance is set to dj (S61). 

[0037]dj branches processing by whether it is below threshold value deltad set up arbitrarily 
(S62). When dj is below deltad, it judges with it being in contact with Pi as QOj from before deltat, 
and the value of Pi is assigned to the portion Qlj applicable to the j-th of the contacting-orders 
storage 16. The flag 2 is stored in fIgQIj (S63). 

[0038]When dj is larger than deltad, it judges with Pi being the newly contacted coordinates, and 
the value of Pi is assigned to Qlj applicable to the j-th of the contacting-orders storage 16 as 
new coordinates. The flag 1 is stored in fIgQIj (S64). 

[0039]After processing of (S63) and (S64) finishes about all the Pi(s), It judges with contact 
having been released about Qlj in which any Pi and a flag are not stored, either, the flag 1 is 
stored in flgQIj, and the value of QOj is assigned to the meaning that the released coordinate 
value is shown at Qlj (S65). If (S65) is processed about all the Qlj, the value of Q1j will be 
moved to QOj (S66). 



[0040]Thus, in the contacting-orders judgment part 15, tlie coordinate value of the 
representation coordinates storage 14 is rearranged into the turn which contacted, and it stores 
in QOj of the contacting-orders storage 16. If processing by the contacting-orders judgment part 
1 5 finishes, processing will be moved to the coordinate information treating part 3. 
[0041]Next, the coordinate information treating part 3 in the first working example is explained. 
The composition of the coordinate information treating part 3 is explained using drawing 8 (a). 
The parameter identifier judgment part 71 determines the parameter identifier assigned to QOj of 
the contacting-orders storage 1 6 based on the contacting-orders-parameter identifier 
conversion table 72, and stores it in the argument storage 75. Furthermore, the value of QOj and 
fIgQOj is also stored in the argument storage 75. 

[0042]The argument value judgment part 73 reads the argument value corresponding to a 
contact position from the position-argument value conversion table 74, and stores it in the 
argument storage 75. The locus primary detecting element 76 stores QOj of the contacting- 
orders storage 1 6 in the corresponding locus storage 77 of contacting orders. 
[0043]When 0 is [ program judgment part 78 ] contained in either of the fIgQOj (1 <-j<-n) of the 
order storage 16 of contact. That is. when one of fingers is released, the program which 
executes a program compatible with the released position based on the locus storage 77 and the 
position-program conversion table 79 is judged, and it stores in the execution program storage 

710. , . ^. 

[0044]The execution check part 71 1 generates the confirmation message shown to a user in the 
form of a dialog box. The coordinates which the center-of-gravity point of two or more points of 
contact may be sufficient as. and contacted eyes most, for example may be sufficient as the ^ 
presentation position of a confirmation message. An argument can be changed when change is 

chosen. ... 
[0045]The output event generation part 712 generates a command script required in order to 
execute a program based on the execute command name of the execution program storage 710, 
and a form and the argument further stored in the argument storage 73, and passes it to 0S5. 
[0046]An example of the position-program conversion table 79 is shown in drawing .8 Xb). The 
execution form passed to an execute command name and a CPU device is made into position 
information and a group, and is stored. A user enables it to set up the position-program 
conversion table 79 freely in the coordinate information set part 5 mentioned later. 
[0047]An example of the execution program storage 710 is shown in drawing 8 (c). The execute 
command name and execution form which have been read in the position-program conversion 
table 79 are stored. 

[0048]An example of the argument storage 75 is shown in drawing 9. The argument according to 
the contacting orders of each finger determined by the argument judgment part 71 is stored in 
the argument storage 75. Furthermore, the value of the coordinate value, QOj, and flag flgQOj 
corresponding to each finger is also stored in the argument storage 75 from the contacting- 
orders storage 1 -6. The file A is assigned to the finger with first contacting orders as the 
argument 1. and it expresses with the example of drawing 9 t hat the coordinates are (90. 90) and 
are the coordinates with which it was in contact from before deltat. 

[0049]An example of the contacting-orders-parameter identifier conversion table 72 is shown in 
drawing 10 . The parameter identifier assigned to the finger of each contacting orders at the time 
of a certain number of contact is stored in the contacting-orders-parameter identifier 
conversion table 72. It means that the argument 1 is assigned at the point of contact which are 
eyes most in case the number of contact is 2. and the argument 2 is assigned at the second 
point of contact in the example of drawing 10 . 

[0050]A user shall set up the contacting-orders-parameter identifier conversion table 72 freely 
using a mouse, a keyboard, etc. Setting out is performed by the coordinate information set part 5 
mentioned later. 

[0051]An example of the locus storage 77 is shown in drawing 1 1 . The coordinates stored in the 
contacting-orders storage 1 6 are divided and stored in the locus storage 77 for every 
corresponding contacting orders. When contact is released, the value of the locus storage 77 is 
cleared. 



[0052] Next, the coordinate information presentation part 4 is explained. In the coordinate 
information presentation part 4, processing which shows a user information, including the 
coordinate value which the user obtained from the coordinate information primary detecting 
element 2 or the coordinate information treating part 3 has pointed out, a parameter identifier 
and an argument value, the program name which it is going to perform, etc., is performed. 
[0053] For example, when a user contacts a finger to an argument icon, its contacting orders, 
parameter identifier, and argument value are shown as a pop up window on the display device 
which hits at the tip of the contacted finger. 

[0054] Drawing 12 is an example of composition of the coordinate information presentation part 4. 
The flag which set whether it would be made to show to which timing to the kind of information 
which can be shown to a user is stored in the presentation information kind storage 111. 
[0055]According to the presentation information kind storage 121, the presentation information 
read station 112 reads required presentation information in the coordinate information primary 
detecting element 2 or the coordinate information treating part 3, and stores it in the 
presentation information storage 1 1 3. Information required for the presentation read by the 
presentation information read station 1 22 is stored in the presentation information storage 1 1 3. 
[0056]The presentation information generation part 1 14 generates the data shown to a user 
based on the data of the presentation information storage 123, passes it to 0S5, and 
presentation is urged to it. 

[0057]An example of the presentation information kind storage 1 1 1 is shown in drawing 13. It is 
stored whether each information, including a parameter identifier, an argument value, a program 
name, contacting orders, etc., which can be shown to a user is shown to a user to the timing (a 
finger is released, when contacting a finger and moving a finger) in which each presentation is 
possible. It means showing a parameter identifier, an argument value, and contacting orders in 
the form where it does not ask for the inputs (O.K./cancellation) from a user like a pop up 
window, when contacting a finger ( drawing 14 ). and not showing a program name in the example 
of drawing 13 . 

[0058]It means showing a parameter identifier and an argument value in the form where it does 
not ask for the inputs (O.K./cancellation) from a user like a pop up window, when moving the 
finger which contacted, and not showing a program name and contacting orders. 
[0059]When the finger which contacted is released, it means showing a parameter identifier, an 
argument value, a program name, and contacting orders like a dialog box in the form where it 
asks for the confirmation inputs (O.K./cancellation) from a user ( drawing 15 ). Processing is 
canceled when cancellation is chosen by the confirmation input from a user. 
[0060] Next, the coordinate information set part 5 is explained. In the coordinate information set 
part 5, GUI for setting up the value of the information 72 required for processing by the 
coordinate information treating part 3, for example, a contacting-orders-parameter identifier 
conversion table, the position-argument value conversion table 74, or the position-program 
conversion table 79 is shown to a user, and the preset value which the user was made to input is 
stored in each conversion table. 

[0061]The start of coordinate information setting out always provides the coordinate information 
setup key in the upper right of the screen, for example like drawing 1 6 (a), and has the method of 
starting, if the button is pushed. Whether the coordinate information setup key was pushed 
judges in the coordinate information primary detecting element 2. 

[0062] Dra win g 16 (b) is an example of a GUI picture for a user to set up the contacting-orders- 
parameter identifier conversion table 72. 

[0063]The program input part 151 for inputting the program which performs argument 
specification, the argument number display 152 which displays the number of an argument 
required for execution of a program, and the argument set part 153 which assigns the kind of 
argument according to contacting orders are displayed. 

[0064]Selection of the program which performs argument specification by the program input part 
151 will display the number of arguments required for the program on 152. The contacting orders 
which are furthermore equivalent to the number of an argument are displayed like 83, and it can 
choose by a list box which argument to assign each contacting orders. If all the setting-out 



items are decided, the execution start button 154 will be pushed, and the result set up by GUI is 
stored in the applicable portion of the contacting-orders-parameter identifier conversion table 
72. 

[0065]Although it assumes that the argument kind assigned to each contacting orders is the 
same in the above-mentioned setting method about the program which needs the argument of 
the same number, It may not be independently the same and what is necessary is in such a case, 
just to extend the contacting-orders-parameter identifier conversion table 72 so that it may 
have a flag which identifies the program to execute. 

[0066]The flow of processing is shown in drawing 17 . To fixed every time deltat, the position 
coordinate which the user's hand contacted in the hand primary detecting element 1 1 is 
detected, and it stores in the hand pattern storage 12 (SI 61. SI 62). It is judged whether the 
coordinate information setting-out start is directed (SI 63). 

[0067]In a coordinate information setting-out start, GUI for setting up the value of the 
contacting-orders-parameter identifier conversion table 72, the position-argument value 
conversion table 74, or the position-program conversion table 79 by the coordinate information 
set part 5 is displayed, and the preset value which the user was made to input is stored in each 
conversion table (SI 64). 

[0068]In the representation coordinates primary detecting element 1 3, the representation 
coordinates of the coordinates with which the contact flag is stored in the hand pattern storage 
12 are defined, and it stores in the representation coordinates storage 14 (SI 65). 
[0069]In not being a coordinate set start, in the contacting-orders judgment part 15, it judges 
whether each coordinates of the representation coordinates storage 14 are contacted by what 
position, and stores a result in the contacting-orders storage 16 (SI 66). 

[0070]In the parameter identifier judgment part 71 and the argument value judgment part 73, the 
argument corresponding to each coordinates of the contacting-orders storage 1 6 is determined, 
and it stores in the argument storage 75 (SI 67). A locus is detected in the locus primary 
detecting element 76, and it adds to the locus storage 77 (SI 68). The program executed by the 
program judgment part 78 is judged, and it stores in the execution program storage 710 (SI 69). 
[0071]In the presentation information read station 112, information required for presentation is 
read and it stores in a presentation information storage (SI 610). Presentation information is 
generated in the presentation information generation part 114 (SI 61 1). Presentation information 
is passed to 0S6 and presentation is performed (SI 61 2). In the execution check part 71 1, it is 
checked whether it performs by the argument which the user set up, and a program (SI 61 3). 
When program execution is directed in the execution check part 711, an output event is 
generated by the output event generation part 712 (SI 61 4). 

[0072] [Modification of the point-of-contact specification method in the coordinate input part of 
the first working example] In the first working example, what was performed simultaneous [ the 
thing performed with the thing performed single hand or both hands or the thing performed by 
one more person / two or more persons ] is available for specification of a point of contact. 
[0073][Modification of the hand detecting method in the coordinate information sensing device of 
the first working example] In the coordinate information primary detecting element 1, the 
mechanism which eliminates the contact coordinate which is not related to an input may be 
formed. For example, when specifying an argument with a finger and the wrist touches the touch 
panel device, the mechanism which eliminates the coordinates equivalent to a wrist is formed. As 
an example of a means to eliminate, for example like drawin g 18 , each contact group's area is 
determined, and when it is more than deltaS with area, there is the method of eliminating the 
group. In drawing 18 . since the area of the field 171, the field 172, and the field 173 is below 
deltas, it is detected, and since the field 1 74 is larger than deltaS. it shall be eliminated. 
[0074]When the contact coordinate more than a definite area is [ the contact coordinate below a 
definite area, and beyond sufficient distance ] separated, the detected contact coordinate may 
be eliminated. 

[0075][Modification 1 of gesture in the coordinate information processing unit of the first working 
example] Although the first working example explained the example which executes a program 
using simple gesture called drag and drop. Gesture may not be restricted to drag and drop and 



may use gesture, such as double-clicking or drawing the seal of O, x, etc. 

[0076] For example, although the program was executed in the first working example by drags and 
drops two or more fingers to which the argument was assigned to a program icon, it may be 
made to execute a program by drawing the seal of O, **, etc. 

[0077]In such a case, what is necessary is just to form the mechanism in which a program is 
identified, into the program judgment part 78 of the coordinate information treating part 3 by 
analyzing the track data stored in the locus storage 77 with a pattern recognition engine. 
[0078]The program to call may be changed according to the kind of drawn seal. For example, 
when two or more files are chosen and O is drawn in the clockwise direction, the program A is 
executed, and when O is drawn in the counterclockwise direction, it may distinguish by the 
method of executing the program B. 

[0079]What is necessary is to make the program corresponding to the position-program 
conversion table 79 with a locus pattern into a pair, and just to store it. A user actually draws a 
locus and enables it to set up setting out of the position-program conversion table 79 by GUI 
like drawing 19 in the coordinate information set part 5. In that case, it can be set up with 
whether the writing order is validated and which finger gesture is performed. Here, the program 
input part 181 for inputting the program which performs argument specification, the argument 
number display 182 which displays the number of an argument required for execution of a 
program, and the argument set part 183 which assigns the kind of argument according to 
contacting orders are displayed. 

[0080]Selection of the program which performs argument specification by the program input part 
181 will display the number of arguments required for the program on 182. The contacting orders 
which are furthermore equivalent to the number of an argument are displayed like 183, and it can 
choose by a list box which argument to assign each contacting orders. Gesture is displayed on 
the auxiliary display 1 84, and if all the setting-out items are decided, the execution start button 
185 will be pushed. 

[0081] [Modification 2 of gesture in the coordinate information processing unit of the first working 
example] Although the first working example explained the example made into the trigger which 
generates a program execution script by drags and drops all the fingers which touch to a 
program icon, It may not necessarily restrict to it, and may drag and drop only by one in the 
finger which touches, and the remaining fingers may be released on the way. As stated to the 
modification 1 of gesture, when using gesture other than drag and drop, gesture, such as drawing 
of a seal, may not be performed with no fingers which contacted, but it may carry out only by 
one of representation. 

[0082]In such a case, what is necessary is just to consider timing which performs program 
detection by the program judgment part 78 of the coordinate information treating part 3 as "the 
time of all the finger being released." 

[0083] [Modification 1 of the argument specification in the coordinate information processing unit 
of the first working example] Although the argument required for program execution, such as 
encryption, was distinguished and specified by contacting orders and it pointed to program 
execution by gesture, such as drag and drop, in the first working example. Input modes, such as 
"writing mode" and "object move mode", may be distinguished by contacting orders. 
[0084] For example, when drawing a figure, there are writing mode describing objects, such as a 
point and a line, and object move mode which moves the drawn object to arbitrary places. 
Although changing these modes conventionally had changed with the menu or the button, an 
input mode may be distinguished by the number of the finger which contacts, concrete — (1) — 
the case where it touches by one — writing mode. It draws with the finger of one flat knot, if the 
finger of one flat knot is released — the end of drawing (2) — the case where it touches by two 
— object move mode. It is object movement at the finger of one flat knot. When the finger of 
two flat knots is released, it is the end of a move. An input mode may be distinguished by the 
number of the finger which contacts as drawing may be then continued with one finger. 
[0085]In such a case, what is necessary is to store the input mode corresponding to the 
contacting-orders-parameter identifier conversion table 72 of the coordinate information 
treating part 3, and just to judge the input mode which refers to it and corresponds by a 



parameter identifier judgment part. What is necessary is just to consider timing which performs 
program detection by the program judgment part 78 of the coordinate information treating part 3 
as "the time of the finger which contacted moving." 

[0086]The [modification 2 of argument specification in a coordinate information processing unit 
of the first working example] Although an argument required for program execution, such as 
encryption, was distinguished and specified by contacting orders and it pointed to program 
execution by gesture, such as drag and drop, in the first working example, Taking advantage of a 
point that gesture can be performed with two or more fingers, package movement of an icon, 
scaling of a window, etc. may be operated with two or more fingers. 

[0087]For example, at scaling of a window, an upper bed and a lower end of a window may be 
pressed down with a different finger, and scaling of a window may be performed by extending 
each up and down, concrete — (1), when an end of a window is touched by one. Gesture may be 
performed with two or more fingers as below scaling (3), three flat knots, and no less than four 
flat knots are the same in two ends to which each touched one touched end when another end 
of the same window was touched with a finger of scaling (2) 2 flat knot. 
[0088] For example, in package movement of an icon, two or more icons may be selected 
simultaneously and each may be rearranged in a desired position. 

[0089]In such a case, what is necessary is to store an input mode corresponding to the 
contacting-orders-parameter identifier conversion table 72 of the coordinate information 
treating part 3, and just to judge an input mode which refers to it and corresponds by a 
parameter identifier judgment part. What is necessary is just to consider timing which performs 
program detection by the program judgment part 78 of the coordinate information treating part 3 
as "a time of a finger which contacted moving." All the above modification is included in this 
invention. 

[0090]It is also possible to realize processing in working example of the invention in this 
application by the program which can be executed by computer, and to realize this program by 
computer as a storage which can be read. 

[0091] As a storage in the invention in this application, a magnetic disk, a floppy (registered 
trademark) disk, A hard disk, optical discs (CD-ROM, CD-R, DVD. etc.). magneto-optical discs 
(MO etc.). semiconductor memory, etc. can memorize a program, and as long as it is a storage 
which a computer can read, the memory form may be which form. 

[0092]OS (operation system) which is working on a computer based on directions of the program 
installed in the computer from the storage. A part of each processing for MW(s) (middleware), 
such as database management software and a network, etc. to realize this embodiment may be 
performed. 

[0093]The storage which the storage in the invention in this application downloaded the program 
transmitted by not only the medium that became independent of a computer but LAN, the 
Internet, etc., and was memorized or stored temporarily is also contained. 
[0094]A storage may be contained in a storage in this invention not only one but when 
processing in this embodiment is performed from two or more media, and composition of a 
medium may be which composition. 

[0095]Computers in the invention in this application may be which composition, such as a 
system by which network connection of equipment which performs each processing in this 
embodiment and consists of one, such as a personal computer, and two or more equipment was 
carried out, based on a program memorized by storage. 

[0096]Apparatus which can realize a function of the invention in this application by a program, 
and equipment are named generically a computer in the invention in this application not only 
including a personal computer but including an arithmetic processing unit, a microcomputer, etc. 
which are contained in an information management system. 
[0097] 

[Effect of the Invention]ln this invention, since the information on two or more fingers that the 
touch-panel screen was made to contact simultaneously in a coordinate information sensing 
device is detectable, it becomes possible to use those locus patterns as gesture. 
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TECHNICAL HELD 



[Field of the Invention]This invention relates to the input device, method, and storage using two 
or more points of contact which operate objects, such as written data and an icon, by contacting 
a display screen with a finger. 
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PRIOR ART 



[Description of the Prior Art]The touch panel is provided with the inputting function which can 
choose objects displayed on the screen, such as an icon and a window, or can draw a 
handwritten character, an illustration, etc. by touching a screen directly with a finger or a pen 
with the display function as a computer display. A finger and a pen can substitute the role of the 
mouse in the conventional desktop computer for a touch panel. 

[0003]It roughly divided into the conventional touch panel, and there were the following two kinds 
of input modes in it. 

[0004]The first input mode is writing mode which writes a character with a finger or draws an 
illustration. The second input mode is the gesture mode in which select an icon with a finger, 
draw the easy form with a finger in carrying out a click and drag and drop, and operation 
corresponding to the motion is performed. There was a thing that a character string will be 
deleted as an example of gesture If a RE seal is written from on a character string by editors, 
such as a memo pad, or the object which is contained in the enclosure if an icon and a character 
string are surrounded by O was chosen etc. 
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EFFECT OF THE INVENTION 



[Effect of the Invention]In this invention, since the information on two or more fingers that the 
touch-panel screen was made to contact simultaneously in a coordinate information sensing 
device is detectable, it becomes possible to use those locus patterns as gesture. 
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TECHNICAL PROBLEM 



[Problem to be solved by the inventionjlt assumed that these conventional gesture drew a line 
with one pen or finger. Therefore, when two or more icons were selected, for example, it had to 
presuppose that it chooses one at a time in order etc., pressing the Shift key by a keyboard, and 
was not able to be operated single hand. Since only one chose the end of the window when the 
size of a window was changed, it has changed in size only into one way at once. 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The entire configuration figure concerning one embodiment of the invention in this 
application. 

[Drawing 2] An example of the coordinate information primary detecting element 2 concerning 
one embodiment of the invention in this application is shown. 

[Dra win g 3 ]An example of the hand pattern storage 12 concerning one embodiment of the 
invention in this application is shown. 

[Drawing 4] The example of composition of the representation coordinates primary detecting 
element 13 concerning one embodiment of the invention in this application. 
[Drawing 5] An example of the representation coordinates storage 14 concerning one 
embodiment of the invention in this application is shown. 

[Drawing 6] An example of the contacting-orders storage 1 6 concerning one embodiment of the 
invention in this application is shown. 

[Drawing 7]It is a figure showing the procedure of the judgment method of the contacting orders 
concerning one embodiment of the invention in this application. 

[Drawing 8] An example of the coordinate information treating part 3 concerning one embodiment 
of the invention in this application is shown. 

[Drawing 9] An example of the argument storage 75 concerning one embodiment of the invention 
in this application is shown. 

[Drawing 10] An example of the contacting-orders-parameter identifier conversion table 72 
concerning one embodiment of the invention in this application is shown. 

[Dr awin g 1 1]An example of the locus storage 77 concerning one embodiment of the invention in 
this application is shown. 

[Drawing 12] It is an example of composition of the coordinate information presentation part 4 
concerning one embodiment of the invention in this application. 

[D rawing 13] An example of the presentation information kind storage 1 1 1 concerning one 
embodiment of the invention in this application is shown. 

[Drawing 14] The example of a pop up window when contacting the finger concerning one 
embodiment of the invention in this application. 

[Dravying 1 5]The example of a pop up window when the finger concerning one embodiment of the 
invention in this application is released. 

[Drawing 16] An example of the coordinate information set part 5 concerning one embodiment of 
the invention in this application is shown. 

[Drawing 1 7] The flow of processing of the whole concerning one embodiment of the invention in 
this application is shown. 

[D rawing 18] The example which eliminates the coordinates equivalent to the wrist concerning 
one embodiment of the invention in this application is shown. 

[D raw ing IJjAn example of the coordinate information set part 5 concerning one embodiment of 
the invention in this application is shown. 
[Explanations of letters or numerals] 
1 — Coordinate input part 



2 — Coordinate information primary detecting element 

3 — Coordinate information treating part 

4 — Coordinate information presentation part 

5 — Coordinate information set part 

6 — OS 
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